Search for: All records

Search-as-learning research has emphasized the need to better support searchers when learning about complex topics online. Prior work in the learning sciences has shown that effective self-regulated learning (SRL), in which goals are a central function, is critical to improving learning outcomes. This dissertation investigates the influence of subgoals on learning during search. Two conditions were investigated: \textsc{Subgoals} and \textsc{NoSubgoals}. In the \textsc{Subgoals} condition, a tool called the Subgoal Manager was used to help searchers to develop specific subgoals associated with an overall learning-oriented search task. The influence of subgoals is explored along four dimensions: (1) learning outcomes; (2) searcher perceptions; (3) search behaviors; and (4) SRL processes. Learning outcomes were measured with two assessments, an established multiple-choice conceptual knowledge test and an open-ended summary of learning. Learning assessments were administered immediately after search and one week after search to capture learning retention. A qualitative analysis was conducted to identify the percentage of true statements on open-ended learning assessments. A think-aloud protocol was used to capture SRL processes. A second qualitative analysis was conducted to categorize SRL processes from think-aloud comments and behaviors during the search session. Findings from the dissertation suggest that subgoals improved learning during search. Additionally, it seems that subgoals helped participants to better retain what was learned one week later. Findings also suggest that SRL processes of participants in the \textsc{Subgoals} condition were more frequent and more diverse. SRL processes that were explicitly supported by the Subgoal Manager seemed to be more frequent in the \textsc{Subgoals} condition as well as SRL processes that were not explicitly supported. 

Researchers in the learning sciences have demonstrated the benefits of effective self-regulated learning (SRL) in improving learning outcomes. The search-as-learning community aims to improve learning outcomes during search, but offers limited research exploring the impact of SRL on learning during search. Current limited research in search-as-learning explores only \textit{perceptions} of SRL processes \textit{after} the search process~\cite{crescenzi_supporting_2021}. Results from such analyses are limited in that SRL is a dynamic, active process and participant perceptions of SRL can be unreliable~\cite{winne_exploring_2002, greene_domain-specificity_2015}. In this paper, we propose the implementation of an SRL coding framework to capture SRL processes as they unfold throughout a search session. Additionally, we offer several implications for future work using the proposed methodology. 

Search systems are often used to support learning-oriented goals. This trend has given rise to the “searchas- learning” movement, which proposes that search systems should be designed to support learning. To this end, an important research question is: How does a searcher’s type of learning objective (LO) influence their trajectory (or pathway) toward that objective? We report on a lab study (N = 36) in which participants gathered information to meet a specific type of LO. To characterize LOs and pathways, we leveraged Anderson and Krathwohl’s (A&K’s) taxonomy [3]. A&K’s taxonomy situates LOs at the intersection of two orthogonal dimensions: (1) cognitive process (CP) (remember, understand, apply, analyze, evaluate, and create) and (2) knowledge type (factual, conceptual, procedural, and metacognitive knowledge). Participants completed learning-oriented search tasks that varied along three CPs (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). A pathway is defined as a sequence of learning instances (e.g., subgoals) that were also each classified into cells from A&K’s taxonomy. Our study used a think-aloud protocol, and pathways were generated through a qualitative analysis of participants’ thinkaloud comments and recorded screen activities. We investigate three research questions. First, in RQ1, we study the impact of the LO on pathway characteristics (e.g., pathway length). Second, in RQ2, we study the impact of the LO on the types of A&K cells traversed along the pathway. Third, in RQ3, we study common and uncommon transitions between A&K cells along pathways conditioned on the knowledge type of the objective. We discuss implications of our results for designing search systems to support learning. 

People often search for information in order to learn something new. In recent years, the “search-as-learning” movement has argued that search systems should be better designed to support learning. Current search systems (especially Web search engines) are largely designed and optimized to fulfill simple look-up tasks (e.g., navigational or fact-finding search tasks). However, they provide less support for searchers working on complex tasks that involve learning. Search-as-learning studies have investigated a wide range of research questions. For example, studies have aimed to better understand how characteristics of the individual searcher, the type of search task, and interactive features provided by the system can influence learning outcomes. Learning assessment is a key component in search-as-learning studies. Assessment materials are used to both gauge prior knowledge and measure learning during or after one or more search sessions. In this paper, we provide a systematic review of different types of assessments used in search-as-learning studies to date. The paper makes the following three contributions. First, we review different types of assessments used and discuss their potential benefits and drawbacks. Second, we review assessments used outside of search-as-learning, which may provide insights and opportunities for future research. Third, we provide recommendations for future research. Importantly, we argue that future studies should clearly define learning objectives and develop assessment materials that reliably capture the intended type of learning. For example, assessment materials should test a participant’s ability to engage with specific cognitive processes, which may range from simple (e.g., memorization) to more complex (e.g., critical and creative thinking). Additionally, we argue that future studies should consider two dimensions that are understudied in search-as-learning: long-term retention (i.e., being able to use what was learned in the long term) and transfer of learning (i.e., being able to use what was learned in a novel context). 

There is a growing body of research in the Search as Learning community that recognizes the need for users to learn during search, but modern search systems have yet to adapt to support this need. Our research proposes three research goals toward addressing the support of user learning during search. Research goal 1 (RG1) introduces a more precise and reliable metric of assessing user learning. Anderson & Krathwohl’s 2-dimensional taxonomy is used as a framework to develop learning objectives and assessment questions to measure user learning during search. Additionally, Anderson & Krathwohl’s taxonomy is used as a coding scheme to outline the pathways users traverse along the way to a particular learning objective. Research goal 2 (RG2) investigates the prediction of learning objectives using behavioral measures. Finally, research goal 3 (RG3) proposes a search system that presents information relevant to the user based on their current learning sub-goal and scaffolds information based on the pathways they are likely to traverse given a particular learning objective. 

In recent years, the “search as learning” community has argued that search systems should be designed to support learning. We report on a lab study in which we manipulated the learning objectives associated with search tasks assigned to participants. We manipulated learning objectives by leveraging Anderson and Krathwohl’s taxonomy of learning (A&K’s taxonomy) [2], which situates learning objectives at the intersection of two orthogonal dimensions: the cognitive process and the knowledge type dimension. Participants in our study completed tasks with learning objectives that varied across three cognitive processes (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). We focus on the effects of the task’s cognitive process and knowledge type on participants’ pre-/post-task perceptions and search behaviors. Our results found that the three knowledge types considered in our study had a greater effect than the three cognitive processes. Specifically, conceptual knowledge tasks were perceived to be more difficult and required more search activity. We discuss implications for designing search systems that support learning.